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ABSTRACT 


The  research  reported  in  this  paper,  conducted  for  the  United 
States  Air  Force  previews  the  experience  gained  from  several  aircraft 
turbine  engine  monitoring  systems  used  over  the  last  decade  and  a  half 
and  examines  the  implications  of  that  experience  for  recently  proposed 
monitoring  systems .  ^  Rand  report  R-2391-AF  and  a  report  now  in 
preparation  describes  in  greater  detail  the  supporting  research. 

-  Two  different  approaches  to  engine  monitoring  have  evolved  in 
attempts  to  achieve  the  goals  of  improved  engine  operations, 
maintenance,  and  management  coupled  with  reduced  maintenance  costs. 

The  first  approach  concentrates  on  day-to-day  operations  and 
maintenance  concerns  and  is  usually  accomplished  by  recording  a  few 
seconds  of  engine  usage  data  cither  at  predefined  performance  windows 
or  when  certain  engine  operating  limits  are  exceeded.  The  second 
approach  focuses  on  long-term,  design-oriented  benefits  that  are 
gained  through  improved  knowledge  of  the  overall  engine  operating 
environment.  Generally,  this  involves  continuous  recording  of  engine 
usage  and  performance.  To  achieve  the  design-oriented  benefits, 
engine  usage  data  must  be  recorded  continuously  on  at  least  a  few 
aircraft  at  each  operational  location.  Our  research  centers  on  six 
examples  of  engine  monitoring  applications  that  indicate  the 
characteristics  and  implications  of  both  approaches. 

Much  uncertainty  still  exists  about  the  benefits  and  costs 
attributable  to  engine  monitoring  systems.  We  believe  that  the 
estimated  maintenance  cost  savings  most  often  used  to  justify  new 
monitoring  systems  are  unlikely  to  materialize  over  the  short  term. 
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But  whether  new  systems  pass  or  fail  in  the  narrow  sense  of  short-term 
cost  savings  should  not  be  the  sole,  nor  necessarily  the  primary  basis 

of  justification.  The  potential  benefits  of  anticipating  needed 
maintenance,  helping  maintenance  crews  and  engineering  support 
personnel  to  better  understand  engine  failure  cause  and  effect,  and 
verifying  that  maintenance  has  been  properly  performed  have 
substantial  value.  These  benefits  can  be  especially  significant  now 
that  the  military  services  are  moving  toward  modular  design  and  an 
on-condition  maintenance  posture  for  new  aircraft  turbine  engines. 
Unfortunately,  none  of  these  potential  benefits  can  be  quantified  on 
the  basis  of  experience  to  date. 
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I.  INTRODUCTION 

Modern  military  aircraft  turbine  engines  present  new  and 
increasingly  difficult  management  and  maintenance  problems  as  their 
level  of  operating  performance,  complexity,  and  cost  continue  to 
escalate.  As  in  the  past,  those  responsible  for  engine  maintenance 
and  management  have  developed  increasingly  more  sophisticated 
techniques  in  order  to  lessen  uncertainty  and  enhance  confidence  in 
their  decisions.  Electronic  monitoring  of  inflight  engine  performance 
represents  yet  another  step,  albeit  a  significant  step,  in  this 
process . 

Central  to  this  concept  is  a  growing  awareness  that  traditional 
maintenance  capabilities  will  not  gauge  with  desired  precision  the 
material  condition  or  performance  of  current  or  future  engines.  This 
is  evidenced  by  the  current  activity  which  finds  some  type  of 
electronic  engine  monitoring  ongoing  or  under  consideration  for  all 
recently  developed  military  aircraft  engines.  But  monitoring  inflight 
engine  performance  isn't  a  new  idea;  during  the  last  two  decades,  at 
least  20  programs  have  been  attempted.  These  range  from  simple  time 
temperature  recorders  to  the  complex  Engine  Diagnostic  System  (EDS) 
being  developed  for  the  F-100  engine  installed  on  the  F-15  and  soon  to 
be  deployed  F-16  tactical  fighters.  Indeed,  while  the  F100  EDS 
program  will  not  be  treated  explicitly  in  this  paper,  it  is  the  reason 
for  our  reviewing  earlier  engine  monitoring  programs.*  This  paper 
presents  results  of  our  analysis  of  six  earlier  monitoring  systems  and 


*See  Reference  1. 
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ongoing  engine  duty  cycle  research  that  are  directly  applicable  to 
current  and  future  engine  monitoring  programs. 


APPROACH  TO  TWS  STUDY 

The  analytial  approach  used  in  this  study  was  to  examine  the 
experience  of  several  selected  engine  monitoring  systems,  with  regard 
to  the  information  and  findings  that  would  be  applicable  to 
consideration  of  new  systems.  The  systems  examined  included:  (1) 

Time  Temperature  Recorder  (TTR) ,  (2)  Engine  Health  Monitoring  System 
(EHMS),  (3)  Malfunction  Detection  Analysis  Recoding  System  (MADARS), 
(4)  Inflight  Engine  Condition  Monitoring  System  (IECMS) ,  (5)  Airborne 
Integrated  Data  System  (AIDS),  and  (6)  Engine  Usage  Monitoring  System 
(EUMS) .*  The  EUMS  is  a  British  system  that  has  a  different  focus  from 
that  of  the  U.S.  systems.  Recent  research  on  engine  duty-cycle 
analysis  was  also  examined .**  Engine  duty-cycle  analysis  is  of 
particular  interest  because  it  can  have  a  significant  effect  on  engine 
design  and  testing.  The  findings  derived  from  both  the  monitoring 
system  experience  and  the  duty-cycle  analysis  will  be  discussed  in 
terms  of  their  implications  for  new  monitoring  systems. 

Recent  monitoring  system  proposals  have  focused  mainly  on  net 
life-cycle  cost  savings,  to  be  obtained  through  reductions  in 
maintenance  labor,  spare -parts,  and  fuel  usage,  as  justification  for 
developing  the  system.***  The  intent  here  is  not  to  challenge  the 

*The  details  of  these  case  studies  are  given  in  Refs.  1  through 

16. 

Ser  Refs.  13,  17,  and  18 

*** 

See  Ref.  19  for  example. 
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cost-estimating  procedure  in  general,  or  any  particular  cost  value 
used  in  life-cycle  cost  calculations,  but  rather  to  focus  on  the 
comprehensiveness  of  the  cost  elements  considered,  the  expectation  of 
achieving  the  net  cost  savings  estimated,  and  the  type  of  information 
presented  to  high-level  decisionmakers  leading  to  selection  of 
particular  systems. 

Design  objectives,  development  plan,  and  life-cycle  costs 
problems  can  arise  with  new  system  proposals  when  the  concept 
formulation  does  not  take  advantage  of  lessons  learned  from  previous 
monitoring  experience.  For  instance,  incomplete  and  potentially 
misleading  estimates  of  life-cycle  costs  can  result.  Thus  a  review  of 

past  systems  histories  is  a  first  step  in  evaluating  current 
expectations . 
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II.  SELECTED  CASE  STUDIES 


The  selected  case  studies,  shown  in  Fig.  1,  reflect  Air  Force, 
Navy,  commercial,  and  British  applications  of  engine  monitoring 
systems.  The  systems  were  applied  to  engines  for  U.S.  military 
fighter,  attack,  trainer,  and  cargo  aircraft,  commercial  transports, 
and  a  spectrum  of  British  aircraft.  Applications  include  both 
single-pilot  and  multi-crew  aircraft  and  single-engine  and  multi-engine 
designs . 


CASE  STUDIES 


SYSTEM 

AIRCRAFT/ENGINE 

COMMENT 

TIME  TEMPERATURE  RECORDER  (TTR! 

SEPT  67  -  JAN  69 

F -100/357 

SINGLE  PARAMETER 
MEASUREMENT;  CONUS  & 

VIET  NAM  EXPERIENCE 
ICONTROL  GROUPI 

ENGINE  HEALTH  MONITORING  SYSTEM 
iEHMS,JULY  76  -  MAY  77 

T -38/J85 

FLEET  RETROFIT  NOT 
RECOMMENDED 
(CONTROL  GROUPI 

MALFUNCTION  DETECTION  ANALYSIS 
RECORDING  SYSTEM  (MADARSI 

LATE  60's  -  PRESENT 

C-5A/TF39 

CONTINUOUS  RECORDINGS 
USED  TO  EXTEND  TF39  TBO 

AND  FOR  CONFIGURATION 
MANAGEMENT 

IN-FLIGHT  ENGINE  CONDITION 
MONITORING  SYSTEM  (IECMS) 

1973  -  PRESENT 

A -7  E/TF41 

USEFUL  FOR  OPERATIONS  & 
COMPONENT  IMPROVEMENT 
PROGRAM:  STILL  IN  DEVELOP 
MENT  (NO  CONTROL  GROUPI 

AIRBORNE  INTERGRATED  DATA  SYSTEM 
(AIDSI 

LATE  60'S  -  PRESENT 

COMMERCIAL 

W 1  DEBOD  1 ES 

LIMITED  APPLICATIONS: 
RETROFIT  NOT  COST 

EFFECTIVE 

ENGINE  USAGE  MONITORING  SYSTEM 
IEUMSI 

EARLY  70’ s  -  PRESENT 

U.K.  AIRCRAFT 

INTENDED  TO  REDUCE 

FUTURE  LCC  THROUGH 
IMPROVING  ENGINE  DESIGN 

Fig.  1 — Case  Studies. 


The  monitoring  systems  themselves  run  the  gamut  of  parameter 
measurement — the  TTR  system  measures  only  a  single  parameter,  whereas 
IF.CMS  measures  in  excess  of  60  parameters.  Most  of  the  systems  use 
snapshot  recording;  some  record  data  continuously.  The  operational 
focus  of  the  monitoring  systems  is  also  varied.  The  U.S.  systems 
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orient  primarily  toward  improving  day-to-day  maintenance,  whereas 
the  British  system  almost  exclusively  emphasizes  the  feedback  of 
operational  data  to  the  design  and  test  communities.  Identified 
control  groups  existed  for  several  of  these  monitoring  systems. 
Unfortunately,  the  control  groups  did  not  control  for  all  the 
variables  of  interest,  and  the  time  intervals  for  most  tests  were  too 
short  to  stabilize  inputs  and  quantify  some  of  the  possible  outcomes. 
Nevertheless,  although  all  the  information  desired  isn't  available, 
much  useful  information  was  obtained. 

The  study  described  here  drew  upon  the  experience  from  all  the 
case  studies,  but  for  this  paper  we  have  selected  two  case  studies  as 
illustrative  examples:  the  EHMS  on  the  T-38,  because  it  represents 
the  predominant  approach  taken  by  the  Air  Force,  and  the  British  EUMS, 
because  the  approach  taken  for  it  is  entirely  different  from  that  of 
the  U.S.  systems. 


THE  T-38  EHMS  CASE  STUDY 

The  T-38  EHMS  emphasizes  improved  day-to-day  engine  maintenance. 
Engine  health  data  are  stored  only  under  the  following  three 
conditions:  (1)  when  engine  parameters  exceed  normal  limits,  (2)  on 
pilot  command,  and  (3)  under  preprogrammed  flight  conditions.  When 
any  of  the  three  conditions  occur,  all  parameter  data  as  of  that 
moment  arc  recorded  (snapshot  recording).  Program  data  were  obtained 
for  the  test  that  was  conducted  from  July  1976  to  May  1977.*  Two 


■'•The  data  for  the  instrumented  and  control  engines  arc  summarized 
by  engine  serial  number  i,n  Ref.  11.  Individual  engine  malfunction  re¬ 
port  data  were  obtained  from  the  Program  office. 
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groups  of  engines  were  used  in  the  test:  an  instrumented  group  and  a 
control  group  (see  Fig.  2).  The  two  groups  contained  the  same  number 
of  engines  and  were  used  for  approximately  the  same  number  of  flight 
hours.  Statistical  testing  detected  no  significant  difference  in  the 
time  since  last  overhaul  or  in  flight-hour  distibutions  among  the 
engines  of  both  groups. 


EHMS  (T-38/J85)  ACTIVITY  OUTCOMES 

JULY  76  -  MAY  '77 


INSTRUMENTED  ENGINES 


NUMBER  OF  ENGINES 

26 

TOTAL  FLIGHT  HOURS 

6226 

MALFUNCTION  REPORTS 

97 

GROUNO  MAINTENANCE 

UNSCHEDULED  REMOVALS 

53 

TROUBLESHOOTING  IMH) 

169 

REPAIR  IMHI 

1403 

ENGINE  GROUND  RUNS 

TROUBLESHOOTING 

52 

TRIMS 

26 

FUEL  USED  IGALI 

TROUBLESHOOTING 

4846 

TRIMS 

5720 

CONTROL  ENGINES 

26 

6443 

48 


23 

90 

530 


38 

14 


1786 

4480 


Fig.  2 — EHMS  (T-38/J85)  Activity  Outcomes. 


Twice  the  number  of  malfunctions  were  reported  for  the 
instrumented  engines  as  for  the  control  engines.  This  resulted  in  a 
larger  number  of  malfunction  reports,  which,  in  turn,  increased  the 
number  of  unscheduled  engine  removals,  troublshooting  and  repair 
manhours,  the  number  of  ground  runs,  and  the  amount  of  fuel  used  for 
troubleshooting  and  trimming  the  engines.  Examination  of  the 
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malfunction  reports  revealed,  however,  that  the  EHMS  flagged  only  five 
malfunctions,  out  of  a  total  of  97,  independent  of  pilot  and 
maintenance  crew  reports.*  From  discusions  with  those  involved  in  the 
test,**  and  an  analysis  of  the  data,  we  conclude  that  the  increased 
number  of  malfunction  reports  resulted  from  increased  sensitivity  on 
the  part  of  the  pilot  and  maintenance  crew.  Both  pilot  and 
maintenance  crew  knew  that  the  system  was  on  the  aircraft,  and  their 
behavior  was  affected  by  that  knowledge.  Although  the  EHMS  reported 
few  malfunctions  independently,  it  did  provide  information  that 
significantly  improved  the  cause-and-ef feet  understanding  of  engine 
problems  encountered,  and  that  was  useful  in  providing  maintenance 
direction  to  the  ground  crews. 

It  is  interesting  to  note  that  the  J85  was  a  mature  engine  and 
that  the  number  of  engine  problems  encountered  was  not  very  great. 
Also,  the  EHMS  was  not  a  new  system,  having  been  through  a  feasibility 
phase  during  December  1972  and  January  1973  Nevertheless,  hardware 
and  software  problems  in  the  EHMS  caused  schedule  delays.  A  6-month 
delay  was  necessary  to  shake  down  the  installation  and  software.  The 
shakedown  not  only  lowered  the  false-alarm  rate  but  also  reduced  the 
EHMS  maintenance  manhours  per  flight  hour  below  the  number  deemed 
necessary  for  day-to-day  operation. 

The  EHMS  Program  Office  had  anticipated  that  less  maintenance, 
less  fuel,  and  fewer  problems  would  result  in  all  the  categories  shown 

*More  recent  data  from  T-38  experience  at  Holloman  AFB,  where  the 
mission  profile  is  more  severe,  indicate  that  the  EHMS  finds  more  en¬ 
gine  problems  independently. 


Ref.  2. 
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in  Fig.  2,  but  more  manhours  and  fuel  were  actually  required, 
resulting  in  higher  cost.  Thus,  confusion  exists  as  to  the  costs  and 
benefits  that  might  reasonably  be  expected  from  an  engine  diagnostic 
system.  Not  all  of  the  potentially  important  system  and  engine 
benef its--improved  readiness,  availability,  reliability,  and  lower 
intermediate  and  depot  maintenance  costs--were  demonstrated  largely 
because  the  controls  needed  to  collect  appropriate  data  for  analysis 
were  not  maintained  over  a  sufficiently  long  term  period.  Current 
objectives  of  monitoring  systems  are  oriented  to  cost-reduction,  and 
the  lack  of  control  for  certain  potentially  important  variables 
results  from  a  lack  of  emphasis  on  non-cost  benefits.  In  spite  of  the 
additional  unscheduled  engine  removals,  maintenance  manhours,  and  fuel 
used,  no  measurable  positive  output--i . e . ,  no  cost  savings  and  no 
increase  in  readiness  or  availability--was  discernible  from  the  EHMS 
experience.  Perhaps  the  engine's  maturity  and  the  shortness  of  the 
test  explain  the  lack  of  positive  outputs. 


THE  BRITISH  EUMS  CASE  STUDY 

The  British  system  (see  Fig.  3)  purposely  ignores  the  short-term 
maintenance-oriented  benefits;  instead,  the  British  choose  to 
concentrate  on  perfecting  their  engineering  understanding  of  the 
engine  operating  environment  and  to  use  this  understanding  to  improve 
present  and  future  engine  designs.  The  system's  primary  objective  is 
to  improve  safety  and  availability  and  ultimately  to  reduce  life-cycle 


cos  t . 
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U.K.  APPROACH  -  ENGINE  USAGE  MONITORING  SYSTEM  (EUMS) 

OBJECTIVES:  -  IMPROVE  UNDERSTANDING  OF 

ENGINE  DUTY  CYCLE /Ml SSION 

APPROACH:  -  INSTRUMENT  SMALL  SAMPLE  OF 

EACH  TYPE  AIRCRAFT 

-RECORD  PARAMETERS  CONTINUOUSLY 
-UTILIZE  DEDICATED  ENGINEERING  STAFF 

-  ANALYZE  DATA  BY  MISSION  TYPE 

EXPECTED  BENEFITS:  -ENGINE  DESIGN 

-CORRELATE  TEST 'DUTY  CYCLES 
-MAINTENANCE 

-  REl  IABILITY 

-COMPONENT  IMPROVEMENT  PROGRAM  ICIPi 


Fig.  3 — U.K.  Approach:  Engine  Usage  Monitoring  System  (EUMS). 


The  focus  of  the  EUMS  reflects  two  conclusions  that  the  British 
reached  in  the  late  1960s  and  early  1970s  when  they  experienced  a 
spate  of  engine  problems  coupled  with  rising  operating  and  support 
costs.  Preliminary  investigations  revealed  that  the  problems  occurred 
because  the  engine's  operational  environment  was  not  completely 
understood.*  Subsequent  studies  concluded  that  a  system  that  would 
provide  a  continuous  recording  of  a  few  parameters  was  the  simplest, 
cheapest,  most  reliable,  and  most  productive  solution  available,  given 
the  prevailing  state-of-the-art.  This  conclusion  was  also  influenced 
by  the  fact  that  despite  the  investments  made  by  the  U.S.  in  complex 


*The  EUMS  is  also  partially  a  result  of  the  difficulty  that  the 
British  had  in  understanding  how  the  Rolls  Royce  Spey  engine  could  be 
so  successful  in  the  F-4K  interceptor,  and  yet  in  a  modified  version 
(the  TF41),  this  engine  could  be  so  troublesome  in  the  A-7E  attack 
a i rcra  f t . 
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monitoring  programs,  there  was  not  a  single  one,  the  British  felt,  had 
explored  the  full  potential  of  engine  health  monitoring.* 

The  British  proceeded  to  develop  a  continuous  recorder  that  would 
record  a  small  number  of  engine  parameters.  For  example,  on  the 
Harrier  engine,  only  eight  parameters  are  recorded.  This  equipment  is 
being  installed  on  a  few  of  each  type  of  aircraft  now  in  operation. 

The  field  data  are  sent  directly  to  a  central  location,  where  they  are 
analyzed  by  a  dedicated  engineering  staff  consisting  of  engineers  from 
the  Ministry  of  Defense  and  from  the  testing  community,  as  well  as  by 
engineers  from  Rolls  Royce.**  Analyzed  by  mission  type,  these  data 
have  yielded  interesting  and  important  observations.  The  British  have 
found,  for  example,  that  the  amount  of  engine  life  consumed  depends, 
to  an  important  extent,  on  the  type  of  missions  flown  and  how  the 
equipment  is  used  during  the  mission.  They  also  found  that  a  major 
contributor  to  reduced  engine  life  is  the  cumulative  effect  of  small 
power  transients.  They  have  concluded--and  the  U.S.  military  services 
are  also  reaching  the  same  conclusion--that  engine  failure  modes,  such 
as  low-cycle  fatigue,  have  not  been  as  well  understood  as  they  were 
thought  to  be.  Quantitative  engineering  data  are  now  being  used  to 
improve  engine  design  specifications  and  to  bring  both  full-scale  and 
component  test  cycles  in  line  with  operational  duty  cycles.  The 
British  are  in  the  process  of  reorienting  their  approch  to 

*Soe  Refs.  6,  7,  and  9  for  a  complete  di cuss  ion  of  the  BUMS  ob¬ 
jectives  and  approach. 

■/.-.'.'The  British  method  of  handling  data  is  designed  for  an  en¬ 
gineering  study.  For  day-to-day  maintenance  purposes  data  must  be 
used  at  the  squadron  level  in  the  field.  This  is  an  important  dis- 
t i net  ion. 
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maintainability  and  reliability,  recognizing  that  these  areas  are  more 
a  function  of  engine  throttle  cycles  experienced  and  the  type  of 
mission  flown  than  of  flying  hours  only. 


ENGINE  MONITORING  SYSTEM  OUTCOMES 

To  evaluate  the  strengths  and  weaknesses  of  previous  monitoring 
systems,  we  divided  system  objectives  into  two  groups  based  on  a  time 
orientation:  (1)  short-term  operations,  maintenance,  and  management 
and  (2)  long-term  design.  Fig.  4  lists  the  characteristics  that  we 
feel  are  desirable  in  a  monitoring  system.  Certain  of  these 
characteristics  served  as  design  objectives  for  each  of  the  case 
studies.  For  example,  as  a  result  of  engine  monitoring,  EHMS  expected 
fewer  maintenance  manhours,  a  savings  in  fuel,  fewer  parts  consumed, 
and  fewer  unscheduled  engine  removals.* 

Since  the  primary  design  objective  of  most  monitoring  systems  is 
to  reduce  resources,  the  five  outcomes  noted  by  checkmarks  in  Fig.  4 
have  received  the  most  attention  because  it  is  easier  to  estimate 
their  costs.  Given  sufficient  information,  all  characteristics  could 
be  assigned  values.  The  first  four  characteristics  have  been 
important  in  providing  justification  for  previous  U.S.  selections,  but 
all  of  them  must  be  considered  because,  in  some  cases,  the  unchecked 
characteristics  may  justify  the  costs  of  a  new  monitoring  system. 

To  show  the  outcome  of  each  monitoring  system,  we  employ  the 
matrix  shown  in  Fig.  5.  The  coding  used  in  Fig.  5  requires  some 
explnnat Ion. 

* 

The  background  of  EllMS  and  details  of  the  benefits  expected  from 
engine  monitoring  are  discussed  in  Ref.  11. 
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SUMMARY  OF  ENGINE  MONITORING  SYSTEMS  OUTCOMES 


MAINTENANCE  ORUNIED 

•  OPERATIONAL 

-  AWARE  Of  ENGINE  HEALTH 

-  AWARE  Of  ENGINE  OVERTLMPERAT'JRES 

•  MAINUNANCl 

''-LESS  MAINTENANCE  MANPOWER 
V-USS  TROUBUSHOOTINC  &  TRIM  FUEL 
—  FEWER  ENGINE  REMOVALS 
'/  -  LESS  PARTS  CONSUMPTION 

—  ANTICIPATE  MAINTENANCE  (TRENDING  1 

-  IMPROVE  CAUSE  &  EFFECT  UNDERSTANDING 
-VALIDATE  MAINTENANCE  ACTION 

•  MANAGEMENT 

MODIFY  TBO 

-  PROVIDE  CONFIGURATION  CONTROL 
DESIGN  ORIENTED 

-  GUIDE  CIP 

-  CORRELATE  TEST /DUTY  CYCLES 

-  AID  FUTURE  ENGINE  DESIGN 


Fig.  4 — Summary  of  Engine  Monitoring  Systems  Outcomes 
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SUMMARY  OF  ENGINE  MONITORING  SYSTEMS  OUTCOMES 


maintenance  oriented 

•  OPERATIONAL 

-  AWARE  Of  ENGINE  HEALTH 

-  AWARE  Of  ENGINE  OVERTEMPERATURES 

•  MAINTENANCE 

-  LESS  MAINTENANCE  MANPOWER 
-LESS  TROUBLESHOOTING  &  TRIM  FUEL 

-  FEWER  ENGINE  REMOVALS 

-  LESS  PARTS  CONSUMPTION 
-ANTICIPATE  MAINTENANCE  (TRENDING) 

-  IMPROVE  CAUSE  &  EFFECT  UNDERSTANDING 
-VALIDATE  MAINTENANCE  ACTION 

•  MANAGEMENT 

-MODIFY  T80 

-  PROVIDE  CONFIGURATION  CONTROL 
DESIGN  ORIENTED 

-  GUIDE  CIP 

-  CORRELATE  TEST  (DUTY  CYCLES 

-  AIO  FUTURE  ENGINE  DESIGN 
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Fig.  5 — Sunmary  of  Engine  Monitoring  Systems  Outcomes. 

The  R  (red)  coding  means  that  the  particular  characteristic  was 
not  achieved;  but  we  must  point  out,  however,  that  in  many  cases  it 
was  not  a  design  objective  and  no  attempt  was  made  to  achieve  it.  The 
Y  (yellow)  coding  signifies  that  the  information  necessary  to 
determine  if  the  monitoring  system  did  or  did  not  accomplish  the 
characteristic  is  lacking.  The  G  (green)  coding  indicates  that  the 
characteristic  was  achieved  or  was  beginning  to  be  achieved.  Several 
of  the  green  blocks  are  crosshatched  to  indicate  that  the  benefits 
existed  but  additional  explanation  is  required. 

Moving  down  the  EMMS  column,  for  example,  we  see  that  both 
operational  characteristics  were  achieved.  Because  overtemperaturc 
damages  the  most  critical  and  most  expensive  engine  components,  this 
information  is  extremely  important.  Our  research  on  engine 
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overtemperatures  in  fighter  and  attack  aircraft  shows  that  pilots 
report  only  about  one-third  of  the  significant  engine  over-temperatures 
and  are  unable  to  report  over-temperature  duration.  This  is  to  be 
expected,  because  these  aircraft  are  mostly  single-crew  aircraft,  and 
the  pilot  sits  in  a  small,  crowded,  vibrating  cockpit  with  a  number  of 
small  gauges.  His  attention  must  be  divided  among  many  functions, 
whereas  the  monitoring  system  watches  engine  parameters  full  time. 

For  the  EHMS,  the  first  three  characteristics  under  maintenance 
are  coded  red  because  the  instrumented  engines  consumed  more 
maintenance  manhours,  troubleshooting,  and  trim  fuel,  and  experienced 
a  higher  removal  frequency,  than  the  control  group  engines  during  the 
flight  test  evaluation.  The  parts  consumption  block  is  colored  yellow 
because  no  data  were  collected. 

The  last  three  characteristics  under  maintenance  are  colored 
green  because  they  were  just  beginning  to  be  achieved,  but  achievement 
requires  time  to  develop  fully.  Looking  at  the  lower  left-hand  corner 
of  the  matrix,  we  see  that  the  management  and  the  long-term  design- 
oriented  blocks  for  both  the  TTR  recorder  and  the  EHMS  are  red. 

Although  the  EHMS  was  not  intended  to  achieve  the  long-term  benefits, 
we  think  that  the  matrix  is  also  telling  us  that  when  a  diagnostic 
system  is  applied  to  a  mature  engine,  it  does  not  have  an  opportunity 
to  make  a  substantial  impact  in  these  two  areas.  This  is  because  the 
engine  is  well  understood  and  most  of  the  problems  have  been 
corrected,  or  its  service  life  is  considered  satisfactory. 

In  contrast,  MADARS,  IECMS,  and  AIDS  were  all  installed  either  as 
original  equipment  or  very  early  in  engine  life.  Each  system  records 
internal  gas  temperatures  anil  pressures.  Since  all  of  them  provide 


-15- 


the  operator  with  an  awareness  of  engine  health,  these  blocks  are 
coded  green.  Under  maintenance,  however,  the  first  four 
characteristics  are  coded  yellow  because  either  all  of  these  engines 
are  instrumented  or  there  never  has  been  an  identified  control  group. 

The  improved  awareness  and  understanding  provided  by  the 
monitoring  systems  has  an  important  effect  of  the  last  three 
maintenance  objectives.  Three  examples  are:  (1)  In  the  case  of  hard 
time  maintenance  procedures,  the  data  can  be  used  to  study  engine 
parameter  trends.  Some  maintenance  problems  can  then  be  alleviated  by 
using  a  different  maintenance  schedule.  (2)  The  data  will  improve  the 

maintenance  crew’s  understanding  of  engine  malfunction  cause  and 
effect,  permitting  the  crew  to  work  smarter  rather  than  harder.  (3) 
The  data  can  be  used  to  avoid  a  shotgun  approach  to  maintenance  in 
which  good  parts  are  needlessly  replaced. 

Unfortunately,  it  takes  time  to  fully  achieve  these  benefits. 
Experiencing  engine  failures,  and  reviewing  the  data  leading  up  to  the 
failures,  precedes  any  attempt  to  correlate  a  particular  engine 
parameter's  trend  with  an  incipient  engine  failure.  Only  when  such 
correlation  exists  can  the  operations,  maintenance,  and  management 
personnel  make  use  of  this  information  to  schedule  or  initiate 
maintenance . 

In  the  case  of  modifying  the  Time  Between  Overhaul  (TBO) 
evaluation  for  the  MADARS,  the  crosshatched  green  block  indicates  that 
the  monitoring  system  data  provide  one  of  many  inputs  that  constitute 
the  information  sets  that  enables  the  TBO  to  be  extended.  The 
technical  data  provided  by  the  monitoring  system  helps  to  advance  the 
TBO  iri  three  ways:  (1)  by  helping  to  uncover  incipient  failure  modes 


that  are  at  present  undetected  in  C-5A  fleet  engines;  (2)  by  tracking 
the  reliability  and  durability  of  new  parts  and  components 
incorporated  into  the  engines;  and  (3)  more,  generally,  by  providing 
additional  confidence  for  the  decisions  affecting  TBO  intervals. 

For  the  IECMS,  the  green  crosshatching  shows  that  the  long-term 
design-oriented  benefits  were  not  part  of  the  original  monitoring 
objectives  but  result  from  an  engineering  need  to  better  understand 
engine  operational  use.  The  IECMS  program  developed  a  continuous 
recoding  option  to  enhance  this  understanding.  The  continuously 
recorded  data  permitted  the  operational  duty  cycle  to  be  correlated 
with  appropriate  testing.  This  is  an  extremely  important 
contribution,  because  unless  correlation  exists ,  operational  fleet 
failure  modes  may  not  be  reproduced  in  the  test  cell . 

The  matrix,  then,  reveals  that  the  U.S.  monitoring  systems  have 
focused  on  the  short-term  maintenance-oriented  benefits.  In  contrast 
to  the  U.S.  systems,  the  British  EUMS  has  ignored  the  short-term 
maintenance-oriented  benefits,  choosing  instead  to  focus  on  the 
longer-term  design,  testing,  and  management  benefits.  The  British 
system  is  different  in  a  way  that  we  think  is  important:  it  focuses 
on  improving  engine  design. 


ENGINE  DUTY-CYCLE  RESEARCH 

The  U.S.  Navy  and  Air  Force,  independent  of  the  British,  have 
also  experienced  many  severe  engine  problems  and,  from  preliminary 
investigations,  have  concluded  that  their  engineering  understanding  of 
the  engine  operational  environment  has  been  inadequate.  For  example, 
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the  type  of  early  information  supplied  to  the  engine  manufacturer  for 
design  guidance  for  a  new  engine  is  shown  in  Fig.  6  for  the  Navy 
F-14.*  The  estimated  power  required  as  a  function  of  time  for  the  F-14 
intercept  mission  is  shown  in  view  (a) .  The  actual  number  of  engine 
cycles  that  occurred  on  an  instrumented  F-14  during  flight  is  shown  in 
view  (b),  together  with  the  resulting  change  in  the  predicted  design 
life  of  certain  important  engine  components.**  Low-cycle  fatigue  is 
important  because  cumulative  fatigue  damage  occurs  in  cyclicly  loaded 
parts  as  these  parts  are  cycled  from  low  to  maximum  RPM.  Current 
methods  for  calculating  LCF  rely  on  the  usage  rates  of  cycles  per  hour 
that  are  derived  from  synthetic  sortie  patterns. 

The  continuously  recorded  flight  data  present  a  different  picture 
for  the  frequency  of  major  and  minor  cycles.  Because  the  increase  in 
the  frequency  of  both  cycles  is  significant  and  was  not  originally 
anticipated,  it  is  contributing  to  engine  problems.  The  data 
demonstrates  the  gross  errors  in  synthetic  sortie  analysis  and 
indicates  that  neither  the  service  nor  the  manufacturer  had  a  clear 
idea  of  the  probable  pattern  of  engine  operational  usage. 

As  a  result,  engine  part  life  has  been  overestimated  and,  hence, 
life-cycle  costs  have  been  underestimated.  By  analyzing  continuously 
recorded  data,  the  original  estimate  of  engine  part  life  was  reduced 
by  an  order  of  magnitude  for  the  first  fan  disk  (see  Fig.  6).  This 


-Propulsion  systems  are  designed  to  Request  for  Proposal  (RFP) 
mission  profiles  based  on  a  projected  combat  time  with  specific 
weapons  load  and  avionics  suite. 

**Thc  imparl  of  engine  duty  cycle  on  part  life  and  maintenance 
intervals  is  discussed  in  Kefs.  17  and  18. 
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type  of  information  is  needed  by  engine  designers  and  is  only 
available  through  continuous  recording.  If,  for  example,  continuously 
recorded  data  had  been  available  from  previous  fighter  aircraft 
mission  experience,  the  F-14  powerplant,  an  uprated  model  of  the  TF30, 
might  have  been  designed  differently. 


-  rfp 
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Fig.  6a — F-14  Power  Required  Profile. 
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Fig.  6b — F-14  Power  Required  Profile. 


-19- 


III.  FINDINGS  OF  CASE  STUDIES  AND  DUTY- CYCLE  EXPERIENCE 

We  now  summarize  the  findings  of  our  case  study  reviews  and  the 
engine  operational  duty-cycle  work.  These  findings,  shown  in  Fig.  7, 
are  as  follows: 

FINDINGS  -  ENGINE  MONITORING  EXPERIENCE 

•  OUTCOMES  FROM  PREVIOUS  ENGINE  MONITORING  APPLICATIONS  ARE 

NOT  CONCLUSIVE 

•  CONTINUOUS  RECORDING  PROVIDES  IMPORTANT  DESIGN  INFORMATION 

•  MODIFICATION  AFTER  SOME  OPERATIONAL  USE  ALMOST  ALWAYS 

DESIRABLE 

•  MONITORING  SYSTEMS  DO  PROVIDE  AWARENESS  Of  ENGINE  HEALTH 

•  MONITORING  TENDS  TO  INCREASE  EARLY  SUPPORT  COSTS 

•  SEVERAL  DEVELOPMENT  PROGRAMS  DOMINATED  BY  HARDWARE  AND 

SOFTWARE  PROBLEMS 

Fig.  7 — Findings:  Engine  Monitoring  Experience. 


(1)  The  benefits  and  costs  of  engine  monitoring  are  still  very 
uncertain.  Many  quantitative  benefits  have  not  been  realized,  and 
costs  have  been  higher  than  expected.  Nonetheless,  because  the  control 
groups  did  not  control  all  the  variables  of  interest,  and  because  the 
time  interval  for  the  tests  was  so  short,  we  can't  be  definitive  about 
the  outcomes. 

(2)  A  continuous  recording  system  provides  important  design 
information  that  can  be  of  substantial  value  to  the  Air  Force, 
although  many  of  the  important  benefits  cannot  now  be  treated 
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quantitatively .  Time  is  required  to  assimilate  this  information  and 
to  develop  and  to  fully  utilize  the  data  derived  from  the  monitoring 
procedure.  Specific  action  will  be  necessary  if  we  are  to  obtain 
certain  long-term  benefits.  The  maximum  utility  of  the  monitoring 
process  occurs  early  in  an  engine's  life  when  an  opportunity  still 
exists  to  affect  engine  component  redesign  and  to  give  direction  to  the 
component  improvement  program. 

(3)  Improvements  in  the  monitoring  system  design  should  evolve 

as  we  gain  experience  with  the  system. 

(4)  Monitoring  systems  provide  the  engine  design  and  test 
community  and  maintenance  crew  with  an  understanding  of  problem  causes 
and  effects  and,  through  corrective  actions,  ultimately  improve  the 
material  condition  of  the  engine.  Engine  over-temperatures  are 
especially  important,  particularly  in  the  case  of  a  single-pilot  and  a 
single-engine  aircraft. 

(5)  The  increased  sensitivity  of  pilots  and  ground  crews  to 
engine  condition  does  result  in  more  malfunction  reports  and 
consequently  increase  costs  initially.  Problems  are  identified  and 
resources  must  be  allocated  to  correct  them.  Another  source  of 
increased  costs  is  the  low  reliability  and  high  false-alarm  rates 
experienced  by  most  of  the  monitoring  systems  during  initial 
operations.  Together  these  two  factors  can  initially  result  in  a  low 
system  credibility,  a  handicap  difficult  to  overcome. 

(6)  Several  of  the  programs  have  been  dominated  by  early 
monitoring-system  hardware  and  software  problems  such  as  latent  design 
deficiencies,  manufacturing  defects,  nonavailability  of  key  software 
subroutines,  and  logic  errors  in  software.  These  problems  continue  to 
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exlst  even  after  the  monitoring  systems  have  reached  the  field.  It 
takes  a  long  time  to  work  out  all  the  bugs,  and  this  drives  up  initial 
support  costs,  especially  when  systems  are  prematurely  fielded.  These 
early  problems  are  difficult  to  overcome  but,  again,  for  both  short¬ 
term  maintenance  and  longer-term  design  and  testing,  the  benefits 
appear  significant  if  they  can  be  resolved. 
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IV.  IMPLICATIONS  FOR  NEW  MONITORING  SYSTEMS 

DESIGN  OBJECTIVES 

What  does  all  this  mean  for  new  monitoring  systems  in  terms  of 
design  objectives,  completeness  of  analysis,  cost  estimates,  and 
development? 

Let's  return  to  Fig.  5.  For  new  designs,  such  as  the  F100  EDS,  we 
would  expect  good  coverage  in  the  operational  areas.  For  the  first 
three  objectives  under  maintenance,  we  would  expect,  initially,  to 
require  more  maintenance  and  fuel,  not  less.  Over  the  long  term,  the 
cost  outcome  is  still  uncertain. 

We  would  expect  to  realize  benefits  that  are  difficult  to 
quantify,  but,  as  we  have  seen  from  all  the  case  studies,  they  will 
take  time  to  develop.  In  the  management  area,  for  example,  it  is  not 
clear  how  the  hard-time  limit  on  the  life  of  critical  parts  will  be 
affected.  As  presently  designed,  new  U.S.  monitoring  systems  are 
snapshot  recording  systems  and,  like  previous  U.S.  monitoring  systems, 
will  not  capture  the  information  needed  for  long-term  design-oriented 
benefits.  Those  benefits  can  be  achieved  by  developing  a  continuous 
recording  option. 


ESTIMATED  COSTS  AND  SAVINGS 

Previous  monitoring  studies  have  focused  on  the  net  life-cycle 
cost  savings.  In  life-cycle  cost  analyses,  the  first  three  cost 

elements — RDT&E,  procurement,  operating  and  support — shown  in  Fig.  8 
are  usually  addressed.  However,  these  elements,  data  management, 
monitoring  system  improvements,  and  modification  costs  associated  with 
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a  specific  system  must  be  added.  The  first  two  additional  elements 
are  generic  to  all  life-cycle  cost  analyses  and  should  be  considered. 
This  problem  is  typical  in  life-cycle  analysis  because,  frequently, 
only  some  of  the  costs  and  some  of  the  benefits  are  analyzed. 


ESTIMATED  COSTS  AND  SAVINGS 


LIFE  CYC  If  COSTS 

-  RDT&C 

-  PROCUREMENT 

-  OPERATING  AND  SUPPORT 

-  DATA  MANAGEMENT 

-  MONITORING  SYSTEM  IMPROVEMENTS 

-  UNIQUE  MODIFICATION  COSTS 

SOURCES  OF  UFE  CYCLE  COST  SAVINGS 

-  FEWER  unscheduled  engine  removals 

-  LESS  FUEL  USAGE 

-  LESS  SECONDARY  ENGINE  DAMAGE 

-  EXTENDED  DEPOT  OVERHAUL 


Fig.  8 — Estimated  Costs  and  Savings. 


We  find  that  with  large,  complex  systems,  resources  required  for 
data  management  can  be  extensive.  We  also  find  that  it  will  be 
necessary  to  make  monitoring  system  improvements  to  increase  the 
initial  low  reliability,  improve  sensor  durability,  and  increase 
system  capability. 

The  users  anticipate  that  the  acquisition  cost  of  the  monitoring 
system  will  be  more  than  offset  by  outyear  engine  support  cost 
savings.  The  first  three  items  shown  on  the  lower  portion  of  Fig. 
8--fewcr  unscheduled  engine  removals,  less  fuel  usage,  less  engine 
secondary  damage-- re  fleet  the  sources  of  these  life-cycle  cost 


savings.  Previous  monitoring  system  experience,  as  shown  by  our  case 
study  reviews,  strongly  indicates  that  the  first  two  items  may  not  be 
realized,  at  least  initially.  Some  data  are  just  beginning  to  appear 
for  the  third  item  (engine  secondary  damage),  and  we  have  been  unable 
to  address  it.  To  these  three  expected  sources  of  life-cycle  cost 
savings,  a  fourth,  not  usually  considered,  should  be  added.  The 
engine  monitoring  system  should  help  to  extend  the  interval  between 
depot  overhauls.  Even  a  small  change  in  this  interval,  affecting  a 
large  number  of  engines,  could  result  in  significant  cost  savings. 

In  addition  to  those  benefits  that  can  be  easily  converted  to 
costs,  full  consideration  must  be  given  to  the  unquantif iable 
benef its--the  unchecked  characteristics  in  Fig.  4 .  For  example,  if  we 
can  improve  our  understanding  of  failure  cause  and  effect--and  thus 
extend  the  depot  overhaul  interval  sooner  than  would  be  possible 
without  engine  monitoring-- reduced  outyear  costs  can  be  achieved  and 
the  system  may  be  cost  effective  over  the  long  term.  There  is  no 
current  evidence  either  way.  The  important  point  is  that  if  cost 
savings  constitute  the  only  criterion  for  monitoring,  is  is  doubtful 
that  the  monitoring  systems  can  be  justified  on  the  basis  of  current 
information. 


DEVELOPMENT  PLAN  CONSIDERATIONS 

Three  important  lessons  learned  from  the  earlier  monitoring 
system  development  plans  arc: 

(1)  A  major  complaint  of  the  earlier  monitoring  systems  programs 
were  that  not  enough  engine  problems  were  experienced  to  wring  out  the 
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system. 

(2)  To  really  be  effective,  more  time  is  needed  in  schedule.  A 
new  monitoring  system  must  operate  as  a  system.  Consideration  must  be 
given  to  operational  software  as  well  as  to  hardware,  i.e.,  to  how  the 
data  are  to  be  used  and  by  whom.  For  maintenance  purposes,  the  data 
must  be  used  at  the  squadron  and  wing  level,  whereas  the  engineering 
data  can  be  processed  at  a  central  site.  If  data  utilization  for 
maintenance  is  not  taken  into  account,  the  time  required  to  become 
fully  operational  will  increase. 

(3)  In  the  process  of  trying  to  model  an  advanced  engine's 
mechanical  and  thermodynamic  behavior,  many  things  will  be  learned 
that  may  cause  a  change  in  the  parameters  monitored  and  thereby  result 
in  modifications  to  the  software  and  hardware.  Thus,  in  all 
likelihood,  several  iterations  of  the  design  will  probably  be 
necessary  until  the  system  functions  as  intended. 


CONCLUSIONS 

There  has  never  been  a  controlled  experiment  of  a  maturing  engine 
over  a  long  enough  time  period  to  allow  monitoring  system  outcomes  to 
be  quantified.  However,  several  conclusions  can  be  drawn  based  on  cur 
case  studies.  These  are  outlined  in  Fig.  9. 

Wc  conclude  that  the  maintenance  cost  savings  used  to  justify  new 
monitory  system  selection  are  unlikely  to  materialize  over  the  short 
term.  But  whether  a  new  system  passes  or  fails  in  the  narrow  sense  of 
cost  savings  over  the  short  term  shoul d  not  be  the  sole  criterion  on 
which  it  Ls  judged .  Substantial  value  lies  in  the  potential  benefits 
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of  (a)  anticipating  needed  maintenance,  (b)  helping  maintenance  crews 
and  engineering  support  personnel  to  better  understand  cause  and 
effect  of  engine  failure,  and  (c)  verifying  that  maintenance  has  been 
properly  performed.  These  benefits  are  especially  significant  now  that 
the  services  are  moving  to  an  on-condition  maintenance  posture.  They 
can  also  be  important  in  helping  to  achieve  the  original  design 
objectives.  Unfortunately,  none  of  these  benefits  can  be  quantified 
on  the  basis  of  experience  to  date. 

CONCLUSIONS 

•  EXPERIENCE  OOEX  NO!  WARRANT  OPTIMISTIC 

NEAR  IERM  EXPECTATIONS 

-  COSTS  ARE  LIKELY  TO  BE  HIGHER 

-  SUCCESSFUL  NEW  SYSTEM  TAKES  TIME  TO  MATURE 

•  TEST  PLAN  NEEDS  TO  YIELD  CONCLUSIVE  EVIDENCE 

ON  VALUE 

-  ADEQUATE  AIRCRAFT  SAMPLE 

■  SUFFICIENT  ENGINE  FLIGHT  HOURS 

-  APPROPRIATE  FOCUS 

•  SCOPE  OF  NEW  SYSTEM  SHOULD  PROVIDE  FOR  VALUABLE 

LONG  TERM  BENEFITS 

-  DESIGN  FEEOBACK 

-  CORRELATION  BETWEEN  TEST  I NG  T  OPER  AT  I ONA  L  USAGE 

Fig.  9 — Conclusions. 

The  development  plan  for  a  new  engine  diagnostic  system  should  be 
designed  to  yield  conclusive  evidence  regarding  its  value.  The  test 
aircraft  sample  and  programmed  flight  hours  must  be  sufficient  for 
selection  decisions.  The  program  focus  must  not  be  too  narrow.  Soft¬ 
ware  and  data  processing  should  be  considered  within  the  overall  system, 


and  the  long-term  design-oriented  benefits  given  proper  consideration. 
It  is  especially  important  to  have  a  balanced  program  now  that  those 
who  decide  on  monitoring  system  utility  must  consider  reliability, 
durability,  and  cost  issues  almost  on  an  equal  footing  with  performance 
objectives. 

* 

To  achieve  the  design-oriented  benefits,  data  for  a  few  carefully 
selected  engine  parameters  must  be  recorded  continuously  on  a  few  air¬ 
craft  at  each  operational  location. 
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